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ABSTRACT 



Two topics are currently being greatly discussed in 
educational literature: (a) summer programs and (b) minority in mathematics 

and science. The Lincoln Foundation summer program provided participants with 
4 weeks of instruction in math and science. The goal of the program was to 
enhance students' ability to succeed in high-school level mathematics and 
science courses. The evaluation design called for the use of a pre- and 
post-test and parent and teacher surveys. The data indicated that the program 
is achieving its objective of increasing students' content knowledge in the 
various subject areas. The results of the parent survey indicated the high 
levels of satisfaction with the program. Results from the teacher survey 
showed overall agreement with the professional development model and 
instructional support. Highlighted curricular methods include: cooperative 
learning; activity-based lessons; and small group techniques. Implications 
for practice and recommendations for improvement are included. (Author/YDS) 
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Abstract 



This process and product evaluation addressed two topics that are currently greatly 
discussed in the educational literature: (a) summer programs and (b) minority in 
mathematics and science. The Lincoln Foundation summer program provided participants 
with four weeks of instruction in math and science. The goal of the program was to 
enhance student’s ability to succeed in high school mathematics and science courses. The 
evaluation design called for the use of a pre- and post-test and parent and teacher surveys. 
As the data in the following report indicate, the program is achieving its objective of 
increasing students’ content knowledge in the various subject areas. The results of the 
parent survey indicated the high levels of satisfaction with the program. The teacher 
survey result showed an overall high support for the professional development model and 
the instructional support. The highlighted curricular methods include cooperative learning, 
activity-based lessons, and small group techniques. Implications for practice and 
recommendations for improvement are included. 

Keywords: Minority Students, High School, Mathematics, Science, Academic 
Achievement, Summer School, Professional Development 
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INTRODUCTION 



Description of the Lincoln Foundation Math and Science Program 

The Lincoln Foundation Summer Math and Science Program is offered to all Jefferson 
County high school students who have completed the eighth, ninth, tenth, or eleventh 
grade. This program was originally designed to enhance gifted minority students abilities 
to succeed in their high school mathematics and science courses. This year the program 
was open to all students entering high school, regardless of race, ethnicity or academic 
standing. 

Located on the campus of the University of Louisville, the program operated on a budget 
of $89,000, funded primarily by Jefferson County Public Schools (JCPS) and served 387 
students (approximately $230.00 per student). The Lincoln Foundation Summer Math 
and Science Program extended over four weeks and included two 90-minute classes 
Monday through Friday. Students having difficulty in their classes were offered one hour 
tutoring sessions following regular classes. The Lincoln Foundation recruited and hired 
10 subject area teachers to provide instruction to the students. The Jefferson County 
Public School system provided five resource teachers who facilitated the professional 
development and curriculum design. The following courses were offered: Algebra 1, 
Algebra 2, Geometry, PreCalculus, Biology, Chemistry, Anatomy/Physiology and 
Physics. The objective of the program was to enhance participant’s ability to succeed in 
their high school mathematics and science courses. 



Program Components 

The Lincoln Foundation program had two basic components: mathematics and science 
instruction for students and professional development for the participating teachers. A 
description of each follows. 
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Mathematics and Science Instruction 

The curriculum was designed to match state mandated core content and to foster 
performance tasks and real life applications of science and mathematics. The ultimate 
objective was to provide instruction that would prepare students for high school 
mathematics and science courses at all levels. For example, the Algebra component 
provided students with experience using manipulatives. Students were required to use 
graphing calculators and graph and analyze original data. The Science courses 
emphasized using inquiry-centered activities for instruction. In Biology, for example, 
science lessons provided students with the opportunity for observation, data collection, . 
reflection and analysis. Other Biology activities encouraged critical analysis of written 
materials. 

Professional Development 

Teachers received a total of 38 hours of professional development during the course of 
the program. Five resource teachers prepared and implemented the professional 
development for the ten core subject teachers. For three days prior to the program, 
teachers met to review the curriculum that the resource teachers had compiled. In 
addition, each afternoon during the four-week program, resource teachers met with 
content teachers for one hour in order to clarify lessons and modify the instruction for the 
following day. 

This format for professional development was modeled after the ‘Rising Star’ 
professional development model. This model for professional development is designed to 
have a long-lasting impact on the teachers. The resource teachers demonstrated and 
modeled hands-on methods of classroom instruction. The teachers were then expected to 
implement these teaching methods with their students in the summer program. It is 
assumed that the teachers will continue to provide hands-on instruction during the regular 
school year to their students. 
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REVIEW OF THE LITERATURE 



To be meaningful, this evaluation must be viewed in the general context of research 
related to (a) minority representation in science and mathematics and (b) educational 
summer programs. Minority students are underrepresented in science and mathematics 
course at every level from elementary to graduate school (Oakes, 1990). Lack of 
preparation in science and mathematics among under-represented minority groups in 
middle and high school ultimately limits college and career choices latter in life (Clark, 
1999). Minority students make up the most rapidly growing portion of our school-aged 
population, but are the population who are most left out of science and mathematics 
programs and course offerings. Research has shown that lack of pre-college preparation 
is the single most important cause of under representation of minorities in science and 
engineering careers (Hayden as cited in Oakes, 1990). There is a need for documentation 
of successful interventions that enhance the science and mathematics performance of 
minority youth. 

The dilemma of how to increase the flow of minority students into the scientific 
"pipeline" has been well documented (Culatta & Gibbons, 1992, Stevens, 1993). To 
date, millions of dollars have been spent on government programs targeted at improving 
minority representation in science and mathematics, but the impact of these programs is 
questionable. In a study by Culatta and Gibbons (1992), seven explanations are provided 
for the weakness of existing programs (a) programs were run with little oversight or 
assessment; funding did not depend on results, (b) there was little real commitment from 
the top or from most faculty, (c) programs had vague or unrealistic goals, (d) funding was 
inconsistent, (e) programs ignored subtle psychological issues such as low expectations 
on the part of teachers, (f) colleges recruited unprepared minority students and then 
provided little support, (g) programs targeted college- aged students or higher, instead of 
going to the root of the problem in elementary, middle and high schools. One widely 
accepted approach to encourage minority participation in science and mathematics is 
through summer programs. 
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Public and private agencies have invested billions of dollars in the past twenty years on a 
wide range of summer programs. Until recently, the majority of school summer programs 
offered to students were designed to provide a pressure-free, noncompetitive environment 
in which young people could explore a particular area of interest in depth (Ware, 2000). 
Currently, there has been a movement towards mandatory summer program designed for 
low performing students (Pipho, 1999). A recent survey of the largest 100 U.S. school 
districts found that 59 had established summer programs to help students who might 
otherwise have been held back; 55 said they had summer classes because they needed to 
meet new state standards; and 17 said they wanted to provide remediation. Only 16 said 
they created the program because they wanted to offer enriched courses. (Borman, cited 
in Mathews, 2000). Critics of summer school programs point out that most programs are 
focused on remedial lessons instead of enrichment (Mathews, 2000). Hahn (1992) 
identified four essentials for quality programs (a) well managed service delivery system, 
(b) effective targeting and recruiting, (c) a well-articulated and consistent identity and (e) 
sound leadership and effective staff. The greatest challenge to summer remediation 
programs is the lack of research showing that such programs work. 
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EVALUATION DESIGN 



Evaluation Objectives 

This evaluation was designed to answer the following questions: 

• What are the characteristics of the- students participating in the program? 

• What are the teacher’s affective responses to different components of the 
program? 

• What are the parent’s affective responses to different components of the program? 

• What is the impact of the summer program on student’s content knowledge in 
science and mathematics? 

Evaluation Model 

Stuffelbeam’s (1971) CIPP model provided the framework under which this evaluation 
was designed. CIPP is an acronym for the four types of evaluation included in this model: 
Context evaluation, Input evaluation, Process evaluation, and Product evaluation. The 
main features of this evaluation include the results from the process evaluation 
identifying how the program was implemented and the short-term product evaluation 
assessing the short-term outcomes of the summer program. 

EVALUATION METHODS 

Participants 

Three hundred and eighty seven students participated in the program. The racial makeup 
of the participants was 84 % African-American, 12% White, 3 % Asian and 2% other. 

The greatest majority of the students (47%) were scheduled to enter 9 th grade in the fall 
of the following year. Twenty five percent were scheduled to enter 10 th grade, 18% to 
enter 11 th grade and 10% to enter 12 th grade. One percent of the students were from ECE 
classrooms. Additional demographic information is provided in Table 1. 
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Instrumentation 

Content Knowledge Pre and Posttest 

Pre and posttests were developed to assess the impact of the program on student’s content 
knowledge. The teachers constructed ten content area tests. The teachers established face 
validity of the instrument. A pretest was given on the second day of the program. A 
posttest was given on the last day of the program. 

Parent student and teacher surveys 

Three surveys were developed to assess the affective responses of the students, teachers 
and parents. Lincoln Foundation program evaluation forms were distributed to parents on 
the Wednesday of the final week of the program. All of the data from the returned 
surveys were entered into SPSS and descriptive statistics were calculated. In addition the 
surveys provided open-response questions, which allowed students, parents and teachers 
to elaborate on their impressions of the program. 
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EVALUATION FINDINGS 



Process Evaluation 

What are the characteristics of the students participating in the program? As described 
in Table 1., this program served largely African American students entering ninth or tenth 
grade. The majority of the students were female (n=205). Approximately one-third of the 
students served participate in the free and reduced lunch program. 

Attendance rates were calculated for each week of the summer program. There was little 
fluctuation in absenteeism during the course of the program. For the first week of the 
camp, 12 % of the students were absent. During the second week of the camp 14% of the 
students were absent and 13% were absent for the third week of the camp. The highest 
absentee rate occurred during the last week of the camp, with 15% of the students absent. 
The average absentee rate for the entire summer program was 13.4%. 

What are the parents of the participant’ s impressions of the program? There was a low 
return rate for the return of parent evaluation forms, with only twenty-one percent (n=85) 
individuals responding. The results are shown in Table 2. More than 87% of the parents 
who responded to the survey felt that the quality of education provided by the Lincoln 
Foundation was excellent. Several parents provided qualitative responses to the survey. 
The two most common responses offered overall praise for the program and criticism of 
disruptive student behavior. The following quotes illustrate this point; “My child had a 
difficult time in math because the students were too noisy... I feel this program is good 
even though she had a difficult time. I would recommend this program” and “My child 
stated the disruptive behavior of several students kept him from learning at times.” 
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TABLE 1. 

Student Profile of the Participants in the High School Math/Science Program 



Variable 


Frequency 


Percent 


Race 


Asian 


9 


2.8% 


Black 


273 


83.5% 


Other 


7 


2.1% 


White 


38 


11.6% 


Gender 


Female 


205 


62.7% 


Male 


122 


37.3% 


Lunch Status 


Free 


67 


20.5% 


Reduced 


40 


12.2% 


Pay 


220 


67.3% 


Grade 


8 th 


152 


46.5% 


9 th 


80 


24.5% 


10 th 


59 


18% 


11 th 


33 


10.1% 


ECE 


3 


.9% 


Type of School 


Private 


23 


7.57% 


Public 


304 


92.43% 



Note : The total enrollment was 395 students. The total number of students with unique 
identification number were 327, representing 83% of the total enrollment. 
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TABLE 2 

Parent Survey Responses 

Question 

1. I believe my child was encouraged to do his/her best work. 

89% Agree 
2% Disagree 

9% Neither Agree nor Disagree 

2. The camp provided a positive learning environment for my child. 

88% Agree 
1% Disagree 

11% Neither Agree nor Disagree 

3. The work my child did in this camp is important. 

92% Agree 
1% Disagree 

7% Neither Agree nor Disagree 

4. I believe my child will be prepared for high school math and science. 

83% Agree 
4% Disagree 

13% Neither Agree nor Disagree 

5. The instructional methods used were effective for my child. 

82% Agree 
4% Disagree 

14% Neither Agree nor Disagree 

6. My child’s interest in science has increased. 

82% Agree 
11% Disagree 

7% Neither Agree nor Disagree 

7. My child’s interest in mathematics has increased. 

50% Agree 
9% Disagree 

21% Neither Agree nor Disagree 

8. The quality of education provided by this program was excellent. 

87% Agree 
0 % Disagree 

13% Neither Agree nor Disagree 

Note : Five-point Likert-type scales were used. Responses indicating 5-4 =agree, 1-2 = disagree and 3 =neither. 
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What are the teacher’s affective responses to the program? Responses to the teacher 
evaluation indicate overall high levels of satisfaction with the program. The following 
results are separated into professional development, instruction strategies used and 
overall assessment of specific components of the program. Teachers indicated that they 
were very satisfied with the professional development offered before and during the 
program. Table 3 details the results from the Professional Development portion of the 
survey. 



TABLE 3 






Survey Results of the Teachers Assessment of Professional Development in the High 

School Math/Science Program (N = 10) 


Item 


M 


SD 


1 . Content was organized to provide opportunities 
to leam new teaching strategies 


4.78 


.63 


2. Materials were adequately covered for understanding 


4.89 


.31 


3. There was opportunity to try instructional strategies 


4.89 


.31 


4. I felt adequately prepared to deliver instruction to 
the students 


4.67 


.47 


Note: Five-point Likert-tvpe scales were used (1 = Disagree, 5 = Agree). The response rate was 67%. The 


reliability coefficient was .61. 







Teacher responses indicated a very high overall rating for the professional development. 
The highest ratings were given to statements related to the adequacy of materials covered 
during the professional development and opportunities to try out instructional strategies. 

Table 4 details the instructional strategies used during the program. The instructional 
strategies most often used included activity-based lessons, cooperative learning groups 
and small group work. The least often used instructional strategy was individual student 
projects. 
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TABLE 4 

Survey Results of the Teachers Assessment of Personal Instruction Strategies in the 
High School Math/Science Program (N = 10) 



Item 


M 


SD 


1 . 


Activity-based approach to presenting lessons 


4.50 


.97 


2. 


Cooperative learning groups 


4.50 


.85 


3. 


Use of manipulatives 


4.20 


1.32 


4. 


Small group work 


4.70 


.67 


5. 


Board work 


3.80 


1.14 


6. 


Individual student projects 


2.40 


1.26 



Note : Five-point Likert-type scales were used ( 1 = Did not implement. 5 = Regularly) . The response rate 
was 67%. The reliability coefficient was .80. 



The means for the overall assessment of the program were generally high. The average 
rating for instructional support, professional development and administrative support 
were very positive. The scores were less positive for the item addressing student 
selection. The teachers gave student selection an average rating of 3.9. More detailed 
information is provided in Table 5 
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TABLE 5 

Survey Results of the Teachers Overall Program Assessment in the High School 

Math/Science Program (N = 10) 



Item 


M 


SD 


1. Student selection 


3.90 


.74 


2. Curriculum 


4.70 


.67 


3. Class size 


4.50 


.71 


4. Length of the program (number of days) 


4.60 


.52 


5. Teaching supplies 


4.70 


.67 


6. Administrative support 


4.67 


.47 


7. Professional development 


4.78 


.42 


8. Instructional support 


4.89 


.31 



Note : Five-point Likert-type scales were used ( 1 - Poor. 5 - Excellent) . The response rate was 67%. The 
reliability coefficient was .76. 



Product Evaluation 

What was the impact of the summer program on participating student’s content 
knowledge in science and mathematics? Statistical analysis of the pre and posttest 
measures indicated students gained knowledge in all eight of the subject areas tested in 
this evaluation. The greatest gain in student knowledge occurred in Pre Calculus. The 
eleven students enrolled in Pre Calculus increased their score by 52% after participating 
in the program. The majority of the students in this years summer program enrolled in 
Biology (n=145). The average increase in Biology test score was 42%. Additional 
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information from all of the subject areas is included in Table 6. These results indicate the 
Lincoln Foundation had a positive effect on the participant’s knowledge of science and 
mathematics subject areas. 



TABLE 6 

Comparison Between Pretest and Posttest Scores For Mathematics and Science 

Subject Areas 



Subject 


N 


Mean 

Pretest 


Mean 

Posttest 


Percent 

Gain 


t value 


Algebra 1 


83 


24.34 


55.87 


31.53 


-19.71 


Algebra 2 


56 


34.39 


53.82 


19.43 


-5.85 


Geometry 


28 


30.42 


45.46 


15.04 


-4.14 


PreCalculus 


11 


15.00 


66.67 


51.67 


-2.68 


Biology 


145 


19.72 


61.31 


41.59 


-26.73 


Chemistry 


47 


52.51 


64.85 


12.34 


-7.83 


Physics 


35 


47.46 


74.29 


26.83 


-11.16 


Anatomy 


12 


32.67 


81.50 


48.83 


-.823 



Note. All subject areas were found to be statistically significant at p< .05 by the paired 
sample t-test. 




16 



15 



CONCLUSION AND DISCUSSION 



Strengths 

An outcome of the program includes gains in content knowledge across all subject areas. 
There were positive difference scores from the pre and posttests for all content areas, 
indicating that Lincoln Foundation Math and Science Program was successful in 
increasing participants content knowledge. The greatest gains occurred in PreCalculus 
(52%), Anatomy (49%), and Biology (42%). 

The curricular methods included cooperative learning, activity-based lessons and small 
group work. Teachers were observed using manipulatives and hands on activities as 
regular instructional strategies. Students were provided with opportunities to engage in 
problem solving activities. This is a hopeful finding in light of research that suggests that 
summer programs often fail because of reliance on traditional teaching strategies and 
teacher centered instruction (Pipho, 1999). 

The rate of attendance (87%) was very high for a summer optional program. This may be 
due to the parent involvement in encouraging students to attend. Parents and students 
were asked to sign a form acknowledging they understood the importance of attendance 
(Appendix A). Parents were also sent attendance records of their children throughout the 
program. Perfect attendance awards were given out at the conclusion of the program. 



Weaknesses 



The main weakness of the Lincoln Foundation program was the lack of measurable 
objectives for the program. The stated objective of the program is to enhance the 
performance of participating students in the science and mathematics courses they will 
take in the fall. There is a need for an operational definition of enhanced performance. It 
is of great importance that the broad statements of purpose of an organization be defined 
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in concrete measurable objectives (Steiner, 1979). In addition, the current stated target 
audience is students entering grades 9 through 12. There is no indication of the student 
ability level that this program is designed to serve. According to Hahn (1992), an 
effective program includes effective targeting and recruiting of students. While this 
program was successful in recruiting students, there appear to be no clearly defined target 
group. 



RECOMMENDATIONS 



Recommendations for Improving the Summer 2001 Program 

• Create measurable objectives for the professional development and student 
program. 

• Clarify the target population and modify the selection procedure accordingly. 

• In order to expedite the evaluation, utilize the scan tron equipment available 
through the Testing office to record pre and posttest data as well as student survey 
results. 

• Provide teachers with more explicit disciplinary procedures. Include 
recommendations set forth by Lincoln Foundation staff in professional 
development of teachers. 
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FURTHER RESEARCH 



This research can be extended in several ways. First, there is much to learn about the 
long-term impacts of this program on student’s achievement in high school mathematics 
and science courses. An additional study is needed to evaluate the long-term impact of 
the program on the participant’s achievement in mathematics and science as measured by 
GPA, scores on the CATS test and enrollment in science and mathematics classes. 

A second direction for future research is to assess the implementation fidelity of the 
“Rising Star” professional development model. One of the objectives of the “Rising Star” 
approach to professional development is that the teachers transfer the knowledge and 
experiences gained during the summer to their own classroom. An additional study could 
examine the teaching practices of Lincoln Foundation teachers in their home schools to 
assess implementation of strategies and materials learned during the professional 
development. A second area of research related to professional development would 
examine the cost efficiency of this model and impact on student achievement. 

A third topic for future research is to examine the effects of the exposure to a university 
campus on the Lincoln Foundation participants. Evidence from the literature indicates 
that summer programs that are offered on college campuses allow students the 
opportunity to test their assumptions about campus life. In addition, College and 
Universities sometimes see summer intervention programs as recruitment devices. An 
additional study could assess the relationship between participation in the Lincoln 
Foundation program and enrollment at the University of Louisville. 
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Appendix A 

Disciplinary procedures as Recommended by Program Staff 
Absenteeism 

3 absences (for any reason) result in automatic dismissal from the program 

Tardiness 

3 tardies (15 minutes or less) - Call home 
6 tardies (15 minutes or less) Dismissal 
1 tardy (greater than 15 minutes) - Call home 
2tardies (greater than 15 minutes) - Immediate dismissal 

Cutting 

1 st cut: Call home 

2 nd cut: Immediate dismissal 



Disrespectful behavior towards staff: Immediate dismissal 
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